Maximum likelihood estimation for a mixture distribution

Fast exact maximum likelihood estimation for mixture of language model

Language modeling is an effective and theoretically attractive probabilistic framework for text information retrieval. The basic idea of this approach is to estimate a language model of a given document (or document set), and then do retrieval or classification based on this model. A common language modeling approach assumes the data D is generated from a mixture of several language models. The...

Maximum likelihood estimation for multivariate mixture observations of markov chains

Penalized Maximum Likelihood Estimation for Normal Mixture Distributions

Mixture models form the essential basis of data clustering within a statistical framework. Here, the estimation of the parameters of a mixture of Gaussian densities is considered. In this particular context, it is well known that the maximum likelihood approach is statistically ill posed, i.e. the likelihood function is not bounded above, because of singularities at the boundary of the paramete...

Constrained Nonparametric Maximum Likelihood Estimation for Mixture Models

A nonparametric mixture model speciies that observations arise from a mixture distribution , R f(x;) dG(); where the mixing distribution, G; is completely unspeciied. A number of algorithms have been developed to obtain unconstrained maximum likelihood estimates of G (see, for instance, Laird 1978, Bohning 1985, and Lesperance and Kalbbeisch 1992), but none of these algorithms lead to estimates...

Penalized Maximum Likelihood Estimation for Univariate Normal Mixture Distributions

Due to singularities of the likelihood function, the maximum likelihood approach for the estimation of the parameters of normal mixture models is an acknowledged ill posed optimization problem. Ill posedness is solved by penalizing the likelihood function. In the Bayesian framework, it amounts to incorporating an inverted gamma prior in the likelihood function. A penalized version of the EM alg...